The present invention relates generally to a tool for verifying the operability of a computer program in the presence of error conditions, and particularly to a system and method for dynamically detecting whether the program explicitly handles a set of defined error conditions.
Computer programs make subroutine calls that perform operations that may fail. Failures may occur because a resource has been over-committed, or because the system is in an unexpected state that is inconsistent with the operation requested. For programs written in some computer languages, failures are turned into xe2x80x9cexceptionsxe2x80x9d that divert the flow of control in the program to an alternative path which reacts to the failure.
However, many computer languages do not provide such exception handling features. For example, C programs do not use exceptions. When using such computer languages, it is the responsibility of the programmer to write procedures so that they return distinguished values, sometimes called error condition values or error return codes, that indicate error conditions. Furthermore, when writing procedures that call a procedure that can return one or more error codes, the programmer is responsible for testing for these error codes explicitly in the calling procedures. If the programmer fails to code the calling procedure so as to detect every possible error condition value that may be returned by each called procedure, an unexpected error may go undetected and lead to the program xe2x80x9ccrashing,xe2x80x9d or corruption of data, or other outcomes less desirable than the program simply stopping.
The task of checking for all possible errors indicated by called procedures is not difficult, but it is tedious, and often neglected. This leads to software that is unreliable, and subject to xe2x80x9cmysteriousxe2x80x9d and difficult to reproduce failures or errors.
It would be desirable to have a tool that identified all error codes that can be returned by a set of called procedures, but which have not been tested for in a calling procedure. One solution is to force each error to occur, and then observe the results. Error codes are artificially introduced. For example, one might rewrite a routine to fail at random intervals. However, there are several drawbacks to this solution. First, since a very large number of errors are possible, it can be a daunting task to test each possible error case. Second, each introduced error code may cause the program to crash, thus requiring a restart after every test.
Even when a program is written in a language that provides an exception handling mechanism, it is often possible for the programmer to fail to specify the exception handling procedures for some of the possible exceptions, leading to crashes rather than more appropriate handling.
One type of program checker, sometimes called xe2x80x9clintxe2x80x9d or a xe2x80x9clint checkerxe2x80x9d determines when a value returned by a called procedure is ignored or discarded by the calling procedure and generates a corresponding warning message. However, lint checkers do not distinguish between a normal return value and an error return value. Therefore lint checkers will not produce a warning when a program uses the value returned by a called procedure, but fails to check whether the returned value indicates an error condition.
In another approach, one can log procedure calls as they are executed. This allows unexpected errors to be detected outside the program. Parasoft""s Insure++ checking tool is an example of a tool designed explicitly for the purpose of detecting unexpected error returns from system calls. However, such checking helps only to discover an error that has occurred and not properly handled by a program, rather than helping to detect whether the program is ready to handle unexpected errors that might one day occur.
In summary, the present invention relates to a tool for detecting unchecked error codes in computer programs. This invention provides a tool that permits a programmer to check whether he has tested for error codes within a program while running the program once or a small number of times. It eases the burden on the programmer by automating much of the checking, and requiring the program to run only as many times as is necessary to cover all the non-error code paths, rather than once per error checked.
The user identifies a number of procedures that can return an error status value, and how the procedures indicate various errors, for example by pushing onto the program stack a return code having a particular value. This information is given to the tool, which instruments the procedures in the program to be tested. When the program being tested is executed, each time that a procedure that could return an error code is called, execution of the program using both a good return value and one or more error return values is simulated.
In one simulation, a non-error value is returned, and the execution path of the program is observed. In each other simulation, a corresponding error code is returned, and the execution path of the program is observed. The execution path of the program for each distinct error code return value of the called procedure is compared with the execution path of the program when the non-error value is returned by the called procedure. If the two paths differ, the program is assumed to handle the corresponding error code. If the two paths do not differ, a report is generated indicating that the program does not appear to properly handle the error condition corresponding to the error code return value.
The simulation process is preferably accomplished by using an interpreter to execute instructions of the program, without affecting the state of the program""s actual address space. This allows the behavior of the program to be examined for each possible error condition, without having to restart the program test for proper handling of each such error condition.
The programmer, or program tester, provides a specification of error code return values for each of a set of procedures that are called by the program to be tested. Whenever a procedure whose error return values have been specified is executed, the tool of the present invention starts interpreting instructions just following the return from the procedure. The interpreter uses a table to store values written to memory by interpreted instructions. Further, whenever an interpreted instruction reads a value from memory or a register, the table is checked to see if it holds the memory location being read. If found in the table, the value stored in the table is used, and otherwise the memory location in the program""s regular address space is read.
During interpretation of the program, system calls that affect state outside the address space are ignored. System calls that modify the address space in unpredictable ways or destroys the address space cause interpretation to terminate. Invalid memory accesses also cause interpretation to terminate.